Natural Language Processing with Small Feed-Forward Networks
نویسندگان
چکیده
We show that small and shallow feedforward neural networks can achieve near state-of-the-art results on a range of unstructured and structured language processing tasks while being considerably cheaper in memory and computational requirements than deep recurrent models. Motivated by resource-constrained environments like mobile phones, we showcase simple techniques for obtaining such small neural network models, and investigate different tradeoffs when deciding how to allocate a small memory budget.
منابع مشابه
A Primer on Neural Network Models for Natural Language Processing
Over the past few years, neural networks have re-emerged as powerful machine-learning models, yielding state-of-the-art results in fields such as image recognition and speech processing. More recently, neural network models started to be applied also to textual natural language signals, again with very promising results. This tutorial surveys neural network models from the perspective of natura...
متن کاملPrediction of the changes in physicochemical properties of key lime juice during IR thermal processing by artificial neural networks
Thermal processing of the key lime juice leads to the inactivation of pectin methylesterase (PME) and the degradation of ascorbic acid (AA). These changes affect directly the cloud stability and color of the juice. In this study, an artificial neural network (ANN) model was applied for designing and developing an intelligent system for prediction of the thermal processing effects on the physico...
متن کاملConvolutional Neural Network for Computer Vision and Natural Language Processing
Machine learning techniques are widely used in the domain of Natural Language Processing (NLP) and Computer Vision (CV), In order to capture complex and non-linear features deeper machine learning architectures become more and more popular. A lot of the state of art performance have been reported by employing deep learning techniques. Convolutional Neural Network (CNN) is one variant of deep le...
متن کاملFeed-forward and Self-organizing Neural Networks for Text Document Retrieval
The aim of this paper is to survey the feed-forward and self-organizing neural networks for the text document retrieval models, which retrieve text documents in a natural language. These models come from linguistic and conceptual approach of the text document analysis, where problems of document representation and document database creation are being solved. The proposed structure of the feed-f...
متن کاملExploring the Depths of Recurrent Neural Networks with Stochastic Residual Learning
Recent advancements in feed-forward convolutional neural network architecture have unlocked the ability to effectively use ultra-deep neural networks with hundreds of layers. However, with a couple exceptions, these advancements have mostly been confined to the world of feed-forward convolutional neural networks for image recognition, and NLP tasks requiring recurrent networks have largely been...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
عنوان ژورنال:
دوره شماره
صفحات -
تاریخ انتشار 2017